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ABSTRACT 

Student response systems, often referred to as “clickers” are small hand-held devices which students may 
remotely respond to questions that are posed during lecture. In this research, the perspectives and lived 
experiences of both instructors and students who used clickers were examined. Also, the activities used by 
instructors were compared to Bloom’s taxonomy levels to provide a new component to our understanding of the 
impact of clickers. Instructors saw clickers as one tool in supporting student learning in their classrooms. The 
improved participation, immediate feedback, impact on attendance, and relatively easy formative assessment that 
result from the use of clickers provided the instructors a method of engaging students. The students requested the 
increase in use because they felt the use of clickers did supported or improved their classroom learning. They 
also enjoyed the peer discussions that instructors facilitated with regard to the use of clickers. Consequently, it 
was found that these systems were especially valuable tool for introductory courses and for monitoring peer 
learning methods in the large lecture classroom. 

INTRODUCTION 

Student Response Systems “SRSs” (Kaleta & Joosten, 2007) or “clicker” (Bergtrom, 2006), as they are 
commonly called, offer a management tool for engaging students in the classroom. Many instructors at both 
large and small educational institutions have begun to use classroom technology that allows students to respond 
and interact via small, hand-held, remote keypads (Caldwell, 2007). 

In order to comprehend the pedagogic developments in this area, it is necessary to understand the practical 
process of using the SRSs. A typical pattern of use is presented in Figure 1. During the lecture, the instructor 
poses a question. Each student has a handset (clicker) that allows students to select the preferred option for the 
answer. The handsets transmit this information to a receiver, which in turn transmits it to the voting software on 
a computer in the class. The handsets transmit to the receiver using wireless technologies, depending on the 
particular system used. After the allotted time, the software produces a histogram or bar chart of the results, 
which is displayed to the students using a data projector to the computer. The instructor then chooses of action to 
respond to the results. The software also allows the data to be recorded so that results can be analyzed later. 






© 


Most classroom response systems 
allow Instructors to export and save 
response data for future analysis and 
evaluation. Some systems also 
Integrate with course management 
systems, like Blackboard, to simplify 
the assessment process. 


Figure 1. Schematic of the Student Response System 
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Most handsets allow multiple-choice responses, with up to ten answers available. The handsets can be used 
anonymously. However, the handsets can be mapped to a student’s name. This allows the instructor to see 
individual's answer, either within the lecture or when reviewing responses at a later stage. 

The key advantage of using the SRS (Student Response System) is that it can give feedback to both students and 
instructors on how well the entire class understands concepts presented. Once this feedback is obtained, an 
instructor can modify the course of instruction, or students can work out misconceptions by peer or classroom 
discussion (Kay, 2009). The SRSs have been used to improve student interaction, engagement, and attention 
(Draper & Brown, 2004; Hinde & Hunt, 2006), increase attendance (Bullock et al., 2002), stimulate peer and 
class discussion (Pelton & Pelton, 2006), provide feedback for both students and instructors to improve 
instruction (Caldwell, 2007), and improve learning performance (El-Rady, 2006; Judson & Sawada, 2002; Kay 
& LeSage, 2009a; Kay & LeSage, 2009b). The fundamental differences between the SRSs and traditional 
classrooms, benefits to using the SRSs, and challenges associated with the SRSs are described in following 
sections. 

Fundamental differences between the SRSs and traditional classrooms 

Feedback can be acquired by multiple means, asking volunteers to share answers, including a show of hands, use 
of small individual whiteboard or tables to display answers, or using colored cards (flashcard) to represent 
multiple-choice responses, in a traditional lecture (Draper et al., 2002; McCabe, 2006; Kay, 2009). 

However these methods have notable drawbacks. A show of hands after students have answered a question for 
the second time is the simplest method. It gives a feel for the level of the class’ understanding and allows the 
instructor to pace the lecture accordingly. The main drawback is a loss of accuracy, in part because some 
students may hesitate to raise their hands and in part because of the difficulty in estimating the distribution. 
Besides, some students are inclined to copy the responses of others. In addition, when hands are lowered, the 
data is lost (Abrahamson, 2006; Burton, 2006; Pelton & Pelton, 2006; Slain et al., 2004). Other shortcomings are 
the lack of a permanent record and the lack of any data collected before convinces his/her neighbor’s discussion 
(Mazur, 1997). 

In contrast to traditional lectures, the SRS-based classroom has several key advantages. The SRS allows students 
to enter their answers to the concept tests as well as their confidence levels, on a variety of handheld devices, 
ranging from graphing calculators to palmtop or laptop computers, which they share in small groups of three or 
four. Their responses are relayed to the instructor on a computer screen and can be projected so the students see 
it, too. The main advantage of the system is that analysis of the results is available immediately. In addition, 
student information is available to the instructor, making large classes more personal; the system can also handle 
numerical and non-multiple-choice questions, and sharing these handheld computers enhances student 
interaction (Al-Fahad, 2009; Beatty, 2004; Hussain & Adeeb, 2009; Keskin & Metcalf, 2011; Mazur, 1997; 
Pradhan et al., 2005). 

Benefits of using the SRSs 

As identified in the introduction section, whilst voting systems can support teaching and learning within lectures, 
any benefits will mostly depend on how effectively they are used on each occasion. In order to judge whether the 
system does, indeed, enhance the lecture format, it is first necessary to identify the assumptions that are made 
about what counts as “good” learning. Three key principles were discussed in the literature; student involvement, 
learning, and assessment and summarized in Table 1. 

_Table 1: Summary of the student response system benefits (Kay & LeSage, 2009)_ 

_Benefit_Description_References_ 

Student 
involvement 

Attendance Students go to class more 

Attention Students are more focused in 

class 


Anonymity All students participate 

anonymously 


Burnstein & Federman (2001); Caldwell (2007); Greer 
& Heaney (2004) 

Bergtrom (2006); Burnstein & Federman (2001); 
Caldwell (2007); d’lnverno et al. (2003); Draper & 
Brown (2004); Elliott (2003); Jackson et al. (2005); 
Jones et al. (2001); Latessa & Mouw (2005); Siau et al. 
(2006); Slain et al. (2004) 

Caldwell (2007); Draper & Brown (2004); Jones et al. 
(2001); Siau et al. (2006); Simpson & Oliver (2007); 
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Participation Students participate with peers 

more in class to solve problems 


Stuart et al. (2004) 

Bullock et al. (2002); Caldwell (2007); Draper & Brown 
(2004); Greer & Heaney (2004); Jones et al. (2001); 
Siau et al. (2006); Stuart et al. (2004); Uhari et al. 
(2003); Van Dijk et al. (2001) 


Engagement 

Students are more engaged in 
class 

Bergtrom (2006); Caldwell (2007); Draper & Brown 
(2004); Latessa & Mouw (2005); Preszler et al. (2007); 
Siau et al. (2006); Simpson & Oliver (2007) 

Learning 

Interaction 

Students interact more with 
peers to discuss ideas 

Beatty (2004); Bergtrom (2006); Caldwell (2007); 
Elliott (2003); Freeman et al. (2007); Kennedy et al. 
(2006); Sharma et al. (2005); Siau et al. (2006); Slain et 
al. (2004); Stuart et al. (2004); Trees & Jackson (2007); 
Van Dijk et al. (2001) 

Discussion 

Students actively discuss 
misconceptions to build 
knowledge 

Beatty (2004); Brewer (2004); Draper & Brown (2004); 
Jones et al. (2001); Nicol &Boyle (2003) 

Contingent 

Instruction can be modified 

Brewer (2004); Caldwell (2007); Cutts (2006); Draper & 

teaching 

based on feedback from 
students 

Brown (2004); Elliott (2003); Greer & Heaney (2004); 
Hinde & Hunt (2006); Jackson et al. (2005); Kennedy & 
Cutts (2005); Poulis et al. (1998); Stuart et al. (2004) 

Learning 

Learning performance 

Bullock et al. (2002); El-Rady (2006); Fagan et al. 

performance 

increases as a results of using 
the SRS 

(2002); Kaleta & Joosten (2007); Kennedy & Cutts 
(2005); Pradhan et al.(2005); Preszler et al. (2007); 
Schackow et al. (2004); Slain et al. (2004) 

Quality of 

Qualitative difference when 

Crouch & Mazur (2001); Caldwell (2007); dTnverno et 

learning 

learning with the SRS (e.g., 
better explanations, thinking 
about important concepts, 
resolving misconceptions) 

al. (2003); Draper & Brown (2004); Elliott (2003); 
Greer & Heaney (2004); Nicol &Boyle (2003) 

Assessment 

Feedback 

Students and teacher like 
getting regular feedback on 
understanding 

Abrahamson (2006); Cline (2006); Draper et al. (2002); 
McCabe (2006); Pelton & Pelton (2006) 

Formative 

Assessment is done that 
improves student 
understanding and quality of 
teaching 

Beatty (2004); Bergtrom (2006); Brewer (2004); 
Bullock et al. (2002); Caldwell (2007); Draper & Brown 
(2004); Dufresne & Gerace (2004); Elliott (2003); Greer 
& Heaney (2004); Hatch et al. (2005); Jackson et al. 
(2005); Siau et al. (2006); Simpson &01iver (2007); 
Stuart et al. (2004 

Compare 

Students compare their the 

SRS responses to class 
responses 

Burton (2006); Caldwell (2007); Draper & Brown 
(2004); Hinde & Hunt (2006); Simpson & Oliver (2007) 


With respect to student involvement, there is considerable data to suggest that students using the SRS are more 
engaged in concepts covered, participate more, pay more attention in class, and are more involved in class 
discussion. One of the greatest of the SRSs is that they offer the opportunity to make the lecture “more 
interactive without appearing threatening”. 

It is likely that many students hold back from answering or responding through peer pressure or the potential 
embarrassment of publicly giving the wrong answer. This in turn may mean that only the more confident or able 
student respond, when they are least in need of instructor attention (Durbin & Durbin, 2006; Fies & Marshall, 
2006; Kay, 2009). 

With respect to learning, numerous studies have reported that students feel they learn more when the SRS is used 
in higher education classrooms (Greer & Heaney, 2004; Pradhan et al., 2005; Preszler et al., 2007). Furthermore, 
many experimental studies have been done where SRS-based classes score significantly higher on tests and 
examinations than classes who are exposed to traditional lecture formats (Kaleta & Joosten, 2007; Kennedy & 
Cutts, 2005; Reay et al., 2005; Reay et al., 2008). 
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Regarding assessment, the SRS helps improve the feedback cycle by ensuring anonymity, collecting, and 
summarizing responses from all students in larger classes very quickly, and limiting the copying of answers 
(Abrahamson, 2006; Beatty, 2004; Draper & Brown, 2004; Pradhan et al., 2005; Simpson & Oliver, 2007). In 
addition, the regular use of the SRS can offer feedback to both the instructor and students as to how well 
concepts are being understood (Bergtrom, 2006; Bullock et al., 2002; Dufrense & Gereca, 2004). Timely 
feedback to students about their performance can be greatly assisted by the use of the SRSs. Because answers are 
marked electronically and automatically, feedback on performance and presentation of the right answers can be 
achieved quickly (Kay, 2009). Students can then see how their performance compares to that of the rest of the 
group. When used for peer assessment students can gain immediate feedback on their work. The instructor can 
also gain feedback in this way. They can, for example, see how well the lecture has performed and use the 
information immediately to provide appropriate action such a re-describing a misunderstood item. 

Challenges associated with the SRSs 

Three categories of challenges were predominant in the literature. Technology, instructor, and student based 
challenges. Each of these challenges is presented in Table 2. 

With respect to technology, on occasion, signals from some remote devices do not register on the instructor’s 
computer, a particularly stressful experience when students are being evaluated for grades. Regarding new 
methods of learning, some students react adversely to the use of the SRS because the overall approach to 
learning changed. They are accustomed to lectures and a switch of methods leads to stress, frustration, and 
resistance at first. Other students are distracted by the use of the SRS (Kay, 2009). 


Table 2: Summary of the student response system challenges (Kay & Lesage, 2009) 


Challenge 

Description 

References 

Technology-based 

challenges 

Bringing remotes 

Students forgot or lost remotes 
and could not participate in 
class 

Caldwell (2007); Reay et al. (2005) 

SRS did not work 

Remote devices did not 
function properly 

El-Rady (2006); Hatch et al. (2005); Sharma et al. 
(2005); Siau et al. (2006) 

Instructor-based 

challenges 

Responding to 
student feedback 

Less experienced instructors 
cannot adjust to student 
feedback 

Abrahamson (2006); Hu et al. (2006) 

Coverage 

Cover less course content if the 
SRS is used 

Beatty (2004); Beatty et al. (2006); Burnstein & 
Lederman (2001); Caldwell (2007); d’lnvemo et al. 
(2003); Burton (2006); Cutts (2006); Draper & Brown 
(2004); Fagan et al. (2002); Freeman et al. (2007); Hatch 
et al. (2005); Sharma et al. (2005); Siau et al. (2006); 
Slain et al. (2004); Steinhert & Snell (1999), Stuart et al. 
(2004) 

Developing 

Time consuming to create the 

Allen & Tanner (2005); Beatty et al. (2006); Boyle 

questions 

SRS questions 

(2006); El-Rady (2006); Fagan et al. (2002); Freeman et 
al. (2007); Horowitz (2006); Paschal (2002); Robertson 
(2000) 

Student-based 

challenges 

New method 

Students find it difficult to shift 
to a new way of learning 

Allen & Tanner (2005); Beatty (2004); Fagan et al. 
(2002); Siau et al. (2006) 

Discussion 

Discussion leads to confusion 
or wasting time 

Draper & Brown (2004), Nicol & Boyle (2003); Reay et 
al. (2005) 

Effort 

Too much effort is required by 
students when using the SRSs 

Trees & Jackson (2007) 
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Summative 

assessment 

Using the SRS for tests may 
not be popular with students 

Caldwell (2007) 

Attendance for 
grades 

Students do not like the SRSs 
used for monitoring attendance 

Caldwell (2007) 

Identifying 

students 

Students want to remain 
anonymous 

Abrahamson (2006) 

Negative 

feedback 

Students feel bad when 
receiving negative feedback 

Carnaghan & Webb (2007) 


The SRS is a new technology and has room for technological improvement and advancement. For example, 
students’ responses sometimes could not be detected and received by the receiver. The receiver was not able to 
receive more than one concurrent response, or the transmitter was not within the range of the receiver. System 
can only capture quantitative data, thus limiting the responses to multiple-choice or true-false questions. Since 
using the wireless handheld transmitter was fun to the students, some of them did not take it seriously-by 
clicking multiple times on purpose, by clicking on answers that were obviously incorrect or by clicking on 
answers that were out of the range/choices given. 

Many researchers have discussed that there are several key problems with current research on the SRSs 
including: a lack of systematic research, a bias toward using anecdotal, qualitative data, excessive focus on 
attitudes as opposed to learning and cognitive processes, and samples derived from limited educational settings. 
Several researchers have maintained that the majority of the SRS data collected to date is anecdotal or qualitative 
(Fies & Marshall, 2006; Kaleta & Joosten, 2007; Schackow et al., 2004). However, both qualitative and 
quantitative data is needed to fully understand the use and impact of the SRSs (Kay & Lesage, 2009a). This 
research provides both qualitative and quantitative results with the statistical analysis obtained from lived 
experiences of both instructors and students. 

METHOD 

In this study provided a unique look at the use of clicker technologies in university classrooms. The perspectives 
and lived experiences of both instructors and students were captured through rich, thick descriptions (Geertz, 
1973). In addition, the actual activities used by instructors were compared to Bloom’s taxonomy levels 
(Anderson & Krathwohl, 2001) to provide an entirely new component to our understanding of the impact of 
clickers. The attitudes of the students were also investigated quantitatively. The quantitative method was probed 
also to elucidate the gender differences in attitudes toward the SRS. 

Four main research questions toward the purpose of the research guided this study were determined as follows: 

1. How do instructors describe their experience of incorporating clickers into their university level 
classes? 

2 . Are instructors who incorporate clickers aware of the cognitive development of their students? 

3 . How do students report their experience of using clickers in their classes? 

4 . How do male and female students differ in their attitudes toward the SRS? 

Participants 

Six faculty members of various academic ranks who were using clicker technologies in their teaching and 
representing a broad range of disciplines, experience with clickers, and academic rank were selected. Firstly, an 
e-mail was sent to each of the instructors outlining the study details and requesting their participation in the 
research. Of the first six instructors emailed, all but two agreed to take part in the study. One pointed out that he 
was not currently teaching a class using clickers but noted a colleague who was teaching that course with 

clickers. When researcher contacted the colleague he agreed to participate. Another instructor did not respond to 
researcher’s e-mail request for participation in the study, and thus another instructor from the list with a similar 
academic rank, discipline, and clicker experience was contacted and did agree to participate. One instructor who 
agreed to participate recommended also including a colleague with several years of experience with clickers. 

Researcher’s final sample of six instructors included two professors (chemical and physics), one associate 
professor (chemical), and one assistant professor (geology), two lecturers (geology and physics). The gender 
distribution was split three female and three male. University and college teaching experience ranged from 8 to 
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37 years and experience using clicker technologies ranged from less than two years to over ten years. The SRSs 
were applied at chemical, geology, and physics engineering departments. 

Once the six instructors were selected and agreed to participate in the study, researcher attempted to recruit two 
students from each instructor's class in which s/he is using clicker technologies. The students were recruited by 
an in-class announcement by the instructor or the researcher. Students were asked to participate in a group 
interview relating to their experience with the use of clickers in that particular classroom. Rather than purposeful 
sampling, students self-selected themselves into the study by responding to the recruitment solicitation. 

These interviews provided triangulation by source and also student opinions about the use of clickers. It was 
intended to secure two students from each instructor’s class to interview as to their perceptions of the use of 
clicker technologies within that course and across campus. A total number of twelve students representing six 
instructors were able to be interviewed. The students represented various academic learning levels, from 
freshman to senior level. Student opinions and feelings about clickers may vary based on their year in school. 
Having only one semester left before graduation and being required to purchase a clicker during that semester 
might play a role in the student’s perception of the use of clickers. However, the hunch was not fully realized 
because of the limited sample size of students researcher interviewed. In addition, students may voice varying 
opinions based-on the number or courses and experiences they have had with clickers. For example, if a student 
has had a number of experiences, s/he may have a different perception of the use of clickers than one who has 
had a very limited experience or only a single, either positive or negative clicker experience. As well, 
interviewing students in even a small group of two may affect their responses. Unfortunately, each student was 
not able to be interviewed individually because of student scheduling and time constraints and this might have 
impacted the results. However, still the interviews of twelve students provided some valuable information 
relating to their opinions and feelings about the use of clickers in the classroom. The student sample including 
six female and six male students consisted four freshman, four sophomores, two juniors, and two seniors. 
Furthermore total number of 523 volunteer students (241 males, 262 females) from the class of the instructors 
supporting the research was given the SRS attitude survey. The student were also freshman (n=135), sophomore 
(n= 129), junior (n=131), and senior (n=128). 

Procedure 

The data of this research was collected with the help of three sources which are interview, observation, and 
survey. 

An e-mail request was sent to those instructors chosen from the list of instructors currently using clicker 
technologies. On agreeing to take part in the study, the instructor participants signed an informed consent 
document and were asked to supply a class period in which researcher could observe them using clicker 
technologies. To see how to design clicker questions and evaluate the benefits and drawbacks of clickers, the 
courses of the instructors supported to this research were observed. This was purely a descriptive and exploratory 
observation during which field notes were taken; researcher served as a non-participatory observer in the class. 
Following the classroom observation, an agreed on date was set to conduct 30 minutes of semi-structured faculty 
interview. Transcriptions of the interview were provided to each instructor participant by e-mail as a member 
check for validity of the transcription. All six participants confirmed the validity of their interview. 

In addition, researcher asked each instructor to provide four sets of classroom slides showing their range of use 
of clickers. The slides were considered document artifacts and analyzed for levels of cognitive process according 
to the revised Bloom’s taxonomy (Anderson & Krathwohl, 2001). The instructors were asked to make an 
in-class announcement to the students for volunteer students. Interested students were directed to contact 
researcher by e-mail for participation and assured that their participation not only would be voluntary, but would 
be anonymous to all parties except the researcher. The students' interviews were audio-taped and transcribed 
using the interview question protocol. They consisted of either a group of 2-3 students or an individual student 
and lasted 25 minutes. Students were also allowed for free-flowing discussion on opinions and thoughts. The 
purpose of the student interview was twofold: to provide a voice to the students as to their perceptions on the use 
of clickers in the classroom and to provide triangulation related to the method and frequency of use of clickers in 
the classrooms represented. Both were partially accomplished through the student interviews. Also to examine 
gender differences in attitudes toward the SRS, the SRS survey was applied to volunteer 523 students at the end 
of the semester. 

Data Analysis 

The phenomenological approach was used in this research. The basis of phenomenology is an interpretive 
paradigm that investigates the qualitatively different ways in which people experience or thinks about something 
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(Marton, 1986). Phenomenography, an approach to educational research that appeared in publications in the 
early 1980s (Marton, 1981; 1986), initially emerged from an empirical rather than theoretical or philosophical 
basis (Akerlind, 2005). Phenomenology begins with an exploration of phenomena, in this case the phenomena of 
using or experiencing clicker technologies in university lectures. Following a traditional qualitative analysis 
approach, data collection and analysis occurred concurrently in this research. 

The contextual experiences of instructors and students are the units of analysis for this study. In addition, 
researcher reviewed clicker slides used by the instructors for cross comparative analysis. This analysis of 
interview data from instructors and students, document artifacts, and observational data allows for triangulation 
by data source and method, thereby increasing the credibility and dependability of the study. Researcher 
transcribed and reviewed the interview narratives for emerging themes relating to the use of clickers to engage 
students in higher-level cognitive thinking. Researcher also reviewed the clicker technology slides and coded 
their contents according to a taxonomy table looking for emerging themes and patterns. A quantitative summary 
of the slide analysis was also formulated. The observation field notes were compared to the student and 
instructor interview narratives and documents supplied by the instructors. Finally, students completed the SRS 
attitude survey at the end of the semester. This survey consisted of nine, seven point Likert scale (from strongly 
disagree “1” to strongly agree “7”) items. Items were constructed based on a review of the SRS literature and 
focused on general attitude, student involvement, learning, and assessment. The internal reliability for the total 
nine-item scale was 0.89 (Kay, 2009). 

Creswell (1998) provided an outline of analysis for a phenomenological study design Researcher followed his 
design which includes a focus on data management, reading and writing memos, description, classification, 
interpretation, and representation or visualization. Data management included recording audiotapes and taking 
field notes. Reading through texts, making notes in the margin, and the formation of initial codes followed. A 
constant comparative method of analysis as described by Glaser & Strauss (1967) was employed. Each statement 
had equal worth and statements were grouped into meaningful units or categories. Finally, a rich, thick 
description (Geertz, 1973) of the data was formulated, with the overall goal to develop a description of what 
students and instructors experience and perceive with regard to the use of clickers for fostering higher-level 
cognitive thinking. 

To ensure rigor and credibility of this study, several verification methods were employed. Creswell (1998) 
provided eight procedures for ensuring the trustworthiness of a study: triangulation, prolonged engagement, 
negative case analysis, clarifying researcher bias, member checks, peer review, thick description, and external 
audits. Creswell (1998) recommended that “researchers must engage in at least two of these in any given study”. 
In this study, researcher used the following strategies as outlined by Creswell: triangulation, prolonged 
engagement, peer review, member checks, thick description, and clarification of researcher bias. 

RESULTS AND DISCUSSION 

The results of the interviews, observations, document artifacts and survey provided by the instructors and 
students were presented in the following sections. 

1. Results of the interviews done with instructors 

Semi-structured interviews based-on the interview question protocol were conducted with each of the six 
instructors. The purpose of the interview and question protocol was to explore the instructors’ methods of using 
clickers and to examine their experiences with using clickers in the classroom. Four themes emerged from the 
six instructor interviews. 

I) There are various uses of clicker technologies for per instructor 

General knowledge questions, problem solving questions included in upcoming exams, questions for formative 
assessment of the class about classroom procedures, questions related to course content that is about to be 
covered with or without point, questions as a review of course content previously covered with or without points, 
questions about high interest or current event topics. 

An example toward using of clicker may be given the course of the lecturer in geology engineering department. 
“As an example of assessing understanding, after teaching the students about the types of geologic faults, I can 
then show them pictures of faults and ask them what type of fault is shown, and type of stress and strain 
produced it. As an example of assessing changes in world-view, I ask the students how old they think the earth 
and the universe on the first day, and then after the geology section of the course I ask them again to see the 
changes in their conception of the age of the earth and universe. I also ask them about their views on evolution 
before the evolutionary lectures and then again afterwards”. 
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II) Instructors view clicker technologies as a tool that enhances classroom learning arid teaching 

A subtheme within this idea of clicker technologies as a tool to teaching and learning was how clicker 
technologies are a tool to pace and modify their current teaching practice. Instructors mentioned that they have 
become much more thoughtful in the design of their lecture presentations. Instructors mentioned how clickers’ 
slides provide for better pacing of the lecture presentations and allow for a reminder of what it is they want to 
discuss that day in lecture. 

Another subtheme that emerged is that i> clicker (www.iclicker.com/dnn/) technology has provided a tool for 
the instructor to think about what questions they want to ask their students during their lectures. The associate 
professor in chemical engineering department declared “ certainly, because the questions associated with clickers 
are conceptually connected to a discipline’s habits of mind, methods and objects of inquiry, ways of 
communicating, ways of knowing, and more”. He told “don’t use them for frivolous questions just to take 
attendance. That will probably make the students resentful of clickers. Ask questions that really pertain to the 
material that has been taught so the students can get immediate feedback on their level of understanding. Also, 
questions on the student’s thoughts before and after being taught a subject can help an instructor tell what 
methods are most effective in learning ”. 

Instructors felt clicker questions generate discussion because students can see immediate feedback as to the 
responses of the entire class. Students know immediately that they are not the only person who got the answer 
correct or who feels a certain way about a topic. Instructors also described using the responses as a means for 
generating further discussion and for modifying their presentation of the content on the spot. The professor in 
chemical engineering department told, “I just like the fact you can get feedback right now, right away, and use 
the teaching moment”. When asked about the benefits of clickers, this idea of immediate feedback for both the 
student and instructor continued to emerge. She continued, “ that they give each student an opportunity to 
contribute his or her ideas, and because the contribution is anonymous to the rest of the class, there is no 
punishment for being wrong. This makes students think more deeply, take a few more risks, and interested in why 
they were off the mark or correct”. 

III) Instructors believe clicker technologies have a positive impact on student engagement and attendance 
Student engagement and increased attendance were commonly mentioned by the instructors interviewed as a 
benefit of clicker technologies. And while none of the instructors interviewed used clicker technologies as many 
their course points or for solely tracking attendance, they saw the benefit that associating points to the questions 
has on student engagement, attendance, and possibly even academic performance. The professor in physics 
engineering department with 40 years of teaching experience explained “7 am sure it has increased 
participation. But the big thing we noticed right away is that the test scores went up. We draw our questions 
from a test bank, so we expect sort of similar performance across the years and last semester was then best 
semester we’ve had for a long-time, long-time in scores for the class. I think the best part is that we think it has 
cleared up questions before the exam". The assistant professor in geology engineering department about 
attendance in the class said “ Students are more likely to come to class if their clickers are being used in some 
way to take attendance or for borderline grade cases as 1 do. The students also enjoy being able to speak-back to 
the instructor by clickers”. The lecturer in physics engineering department told “ positively, the questions focused 
their attention on particularly important issues”. The lecture in geology department declared “7 have not 
measured attendance before and after the use of clickers so it is hard to tell. I get a sense that a slightly larger 
proportion of students attend since I started using clickers”. 

Another subtheme is the clicker technology which has had student engagement in the classroom and therefore 
impacted their teaching of the content. Instructors noted active engagement with the content, a decrease in 
student incivility, and occasionally, an increase in student attendance. The professor in physics engineering 
department stated “ students seem more engaged. To my mind as I lectured, they seemed more engaged and I am 
even asking more concept questions, you know also clicker questions”. 

IV) The benefits and drawbacks of using clicker technologies 

Instructors noted technical issues and limits they encounter with clicker technology, but almost all cited personal 
error as the main reason for technical glitches they encounter. Instructors also cited frustration on the part of 
students and disappointment when technical errors occur in the classroom session. The lecturer in physics 
engineering department explained they are difficult to use with open-ended questions and could squelch class 
discussion. They could also place too much focus on being correct rather than understanding why one is {or is 
not) correct”. She added, “ you only get five choices for responses. With more subtle subjects, that may not be 
enough”. On the other hand according to the associate professor in chemical engineering department, “ Overall, 
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I have had a lot of fun with them. I have hoped that they have improved my teaching, overall they have helped 
improve my accessibility to the students because they ask more questions at the end of class. They are hard to 
get out of the room at the end of class. I think it is because they know I am interested in their responses, opinion 
questions, and they know I am interested”. 

The faculty interviews were filled with thick description of the clicker technologies, their teaching, and 
eventually the classroom learning environment. It was interesting to note that none of the instructors felt they 
had reached the pinnacle of clicker understanding and use. Each one mentioned the process as evolving and 
thought they could do more with the technology to improve student learning. However, another subtle theme that 
emerged through the overall interview process was the student-centered approach each of these instructors held. 
While researcher was not able to distinguish whether that approaches was held before the use of clickers or at 
their onset of using clickers. They seemed to feel that clickers allowed for this improvement to learning, not only 
by engaging students and soliciting student feedback, but also by changing the way they designed and carried out 
the lecture itself. Instructors went into their class sessions open to hearing what students knew and thought about 
content, open to the discussion that might be generated based on clicker questions, and open to modifying 
previously held ways of teaching in an attempt to improve the classroom environment for the students they 
teach. And in the process, the instructors realized this was not only beneficial but also fun. 

2. Results of the researcher’s observation toward using clicker 

As a part of this study, an observation of each of the six instructor participants was conducted to view how they 
were using clickers in the classroom. Researcher served as a non-participatory observer, generally sat in the rear 
or middle of the classroom, and transcribed field notes as to happenings in the classroom. The observation was 
structured in such a manner so that the instructor did not change his/her normal teaching and use of clickers 
based on participation in the study. From the observation field notes, researcher conducted a cross-comparative 
analysis, looking for emerging themes on the observations of instructors. Four themes emerged from the 
observation field notes. 

I) Full classroom discussion is initiated following clicker questions. 

The observation revealed that almost each instructor encouraged students to hold a peer discussion relating to 
most of clicker questions posed. This peer discussion was fostered either before answering clicker question or 
after students answer clicker question but before showing the polling results. It was exciting to witness the 
explosion of student discussion given this opportunity. As an observer, it seemed the students were on target 
with their discussion and these discussions took only a limited time to complete in the overall time in the 
classroom. 

II) Various technological tools are used in the classroom. 

The instructors using clickers also used various other technological tools in the classroom. Instructors used 
PowerPoint, Graphics, Images, Video, etc., which is a necessity to the i>clicker technology (Fig. 2), was being 
used by each instructor. This combination of learning systems enhanced the engagement of the students in the 
class. 



Figure 2. Student using i>clicker in the class 


III) Most students had their clickers and were taking part in. 

In the beginning of the course a headcount of the number of students present in the classrooms was conducted. 
This manual count was compared with the number of response registering in i>clicker technology. In every 
observation instance it appeared most of the students present were participating in clicker questioning a 
response. 
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Most students in this study seemed open and willing to use clickers and even desired greater use of clickers 
across campus. All students, regardless of their like or dislike of clickers, were able to identify benefits of 
clickers to their learning, given the instructor was component in clicker technology. 

The only halting point was the technical issues that instructors encountered. Therefore the technical and 
pedagogical education instruction should be provided to instructors before implementation of clickers in the 
classroom, thus alleviating some of the student and instructor frustration with technological errors on the part of 
the students and the instructors. 

In addition students often want to know why clickers are being used, how often clickers will be used, and what is 
expected of them on the use of clickers. While clicker technologies are relatively user-friendly, campuses should 
make technical assistance available to students. Instructors should also gain enough understanding of the 
technology to provide small technical assistance in the classroom and to explain their reasoning behind their use 
of clickers in the classroom. 

3. Results of the slide’s analysis provided by instructors 

As a part of their participation in this research, instructors were asked to supply several sessions of their 
i>clicker slides to be compared to Bloom’s taxonomy of cognitive domains. The slides they submitted were at 
the discretion of the instructors. Several instructors sent a range of their use of clickers, and they were free to 
select the slides to be shared with me for analysis. 

The purpose of the slide analysis portion of the study was multidimensional. It served as a method of 
triangulation to support the observation and interview results as to how often, and in what manner, clickers were 
being used in the classroom. It also allowed for an analysis of the cognitive level of use of clickers in relation to 
Bloom’s taxonomy. 

On receiving the slides, the number of times was counted in which a clicker question was asked per session, and 
calculated the total number of slides presented in that session. Overall, researcher analyzed 12 sessions of slides 
included 143 clicker questions slides, provided a somewhat quantitative aspect to the study. Table 3 outlines the 
slide analysis results. Also sample clicker questions toward the courses were represented in Figure 3. 

The six levels of Bloom’s taxonomy, from the least complex to the most complex of thinking, include: 
knowledge, comprehension, application, analysis, synthesis, and evaluation. Each slide was placed within one of 
the six levels of Bloom's taxonomy of cognitive domains with instructors. When a question was difficult to 
place, it was placed in the lower of the possible levels rather than in a higher-level. Consistency was a key to 
analyzing the slides. 


Table 3: The number of slides related to Bloom’s Taxonomy 


Courses 

Knowledge 

Comprehension 

Application 

Analysis 

Synthesis 

Evaluation 

Physics 

17 

12 

19 

2 

2 

3 

Chemistry 

16 

4 

27 

3 

4 

4 

Geology 

8 

- 

14 

- 

- 

8 

Total 

41 

16 

60 

5 

6 

15 


Note: Each session was 90 minutes and each course was observed four times. The physics and chemistry lectures 
were represented average 14 slides in each session. Also geology lectures were presented the average 8 slides. 
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-c- 

Figure 3. Examples of applied reasoning questions presented in class with the SRS for (a) Geology, (b) Physics, 

(c) Chemistry courses. 

What became evident in the analysis of the slides is the variety of use by instructors both within and across the 
disciplines represented. Recognizing this variety of use by instructors provided triangulation with the results 
from the observation and interviews, substantiating the idea that these instructors pose clicker questions in a 
variety of ways in the classroom. 

It is interesting to note the large number of slides used in several of the classroom sessions. The classroom 
session slides analyzed did include most slides, some of which had only one minor difference from the slide 
before or after it. Instructors might also have used a single slide to include a graphic or a website link. It may 
also be that these instructors included a plenty of slides they may never actually get through in the designated 
class period and then carry over to the following class period. Figure 4 indicates the graphical representation of 
the total number of slides per domain. It is obvious that most of clicker slides analyzed fell into the application 
domain with the second most being knowledge level slides. 
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Analysis of Slide to Bloom's Taxonomy 



Bloom's Domain 


Figure 4. Analysis of the slides according to Bloom’s Taxonomy 


It was difficult to create higher-level thinking within multiple-choice questions; however it appeared with the 
thoughtfulness involved in asking clicker questions, these instructors have begun to foster higher-level of 
cognitive thinking. It was believed that this critical thinking was fostered not only in the manner in which the 
instructors posed the questions, but also in the discussion the instructors initiated after the question was posed. 
While knowledge, comprehension, and application type questions were most of the slides presented by clicker 
technologies, the level of cognition required increased as instructors forced students to make a decision, to stick 
to that decision, and to discuss that decision with peers. 


The instructors also should pay attention while preparing the single question and question sequences according 
to Bloom’s Taxonomy. The potential drawback of relating the different type of questions could be avoided by 
preparing a single question and question sequences. A single question usually fails to help students make 
context-dependent connections. Question sequences have three or four questions, each with a context that looks 
different to students, while the underlying concept looks equivalent to experts. By recognizing and applying a 
new concept in different contexts and conditions, students can obtain a better level of understanding according to 
a single question. By using question sequences, instructors can have a better understanding of where the 
students’ difficulties are, and thus can provide corresponding feedback. Question sequences can also provide 
specific feedback to students themselves. A common difficulty when students learn science courses is that they 
cannot identify their mistakes. Question sequences can help students find specific difficulties. So, question 
sequences can create cognitive with less anxiety (Reay et al., 2005). 

4. Results of the interviews done with students 

The students had varying levels of experiences with clickers, from freshman to senior classes. Their opinions 
were thought-provoking, encouraging, and concerning. Four themes were emerged from the student interviews: 

I) Students are able to identify benefits to their learning relating to the use of clickers in the classroom 

As a part of the interviews, students were probed as to the methods used by their instructors, the benefits and 
limits of clickers, and the impact of clickers on learning. All students, whether they expressed a like or a dislike 
for clickers, were able to generate benefits from the use of clickers. Students liked the classroom and individual 
feedback that clickers provided. They liked being able to see potential exam questions before the exam as well as 
being able to know immediately if they were correct in their answer. Students also liked the interaction with 
other students that their instructors encouraged with regard to clicker questions. Finally, some students felt that 
clickers improved their learning and added to the level of cognitive thinking required in the classroom. 

II) Students value technical competence in the instructors who use clickers. 

Three students, all from one class where the instructor is a new clicker user, were frustrated with the level of 
technological competence exhibited by their instructor. Technical errors on the part of their instructor frustrated 
students, especially when the instructor used class time to try to resolve the errors. A senior student in geology 
engineering department recommended that instructors be required to take a clicker class to learn how to use the 
technology before implementing them in the classroom. 

III) Students would like to see more instructors use clickers 

Students expressed a desire for more use of clickers across campus and within their individual courses. This 
desire seemed to be expressed by both those students who felt their instructor was using clickers adequately, in 
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that they would like other instructors to use clickers, and also by those students who felt their instructor was not 
using clicker technology enough in their specific course to justify the cost. 

TV) Students would like information on the use of clickers 

Students mentioned the need for the campus or individual departments to list which courses are currently using 
clickers or are most likely to be using clickers in the future to allow students to know if they should keep their 
clickers or sell them. In addition, students noted that simply having an instructor tell them to keep their clicker 
for the remainder of their time on campus would be a benefit since they might need to use it in another class. 

Besides, students wanted to know what content their clicker quizzes would cover in the event points are 
associated with correct and incorrect answers. Also students wanted to know what was expected of them 
regarding their use of clickers and what will be considered academic dishonesty as it relates to the use of 
clickers. Students mentioned abuse by students who are giving their clicker to another student to click in for 
them during class. When asked if they felt this was academic dishonesty, most of students mentioned that it 
would depend on whether the instructor had stated it as such in the beginning of the semester or in the syllabus. 

5. Results of the SRS attitude survey according to gender 

With respect to total the SRS attitude score, male students (M=56.5, SD=9.2) had significantly more positive 
attitudes towards the SRS than female students (M=52.1, SD=10.1) (t=5.2, df=521, pc.001). The effect size of 
0.45 is considered to be in the medium range by Cohen (1988). Since overall attitudes toward the SRS were 
significantly different, a MANOVA was run to compare male and female students on each of the nine-Likert 
scale survey items examining attitudes toward using the SRS. Hotelling’s T was significant (pc.OOl), so 
individual comparisons were done on each survey question. Male and female students differed significantly on 
all items (Table 4). 


Table 4: Gender differences in attitudes toward the SRS based on survey results 


Measure 

Males 

Females 

F 


M 

SD 

M 

SD 


Overall Attitude 

When a SRS was used, the class was better 

5.07 

1.22 

4.41 

1.35 

28.01* 

Student Involvement 

I was more engaged in the lesson when a SRS was used 

5.44 

1.32 

5.12 

1.52 

9.31** 

I was more motivated when a SRS was used 

5.43 

1.41 

4.99 

1.52 

19.28* 

I participated more than I normally would when a SRS was used 

5.61 

1.33 

4.97 

1.47 

7.55*** 

Using a SRS generated more class discussion 

4.98 

1.46 

4.34 

1.53 

18.92* 

Learning 

I learned more when a SRS was used 

5.02 

1.48 

4.65 

1.69 

12.75* 

Assessment 

Using a SRS was a good way to test my knowledge 

5.67 

1.35 

5.19 

1.48 

7 gl*** 

I liked seeing what other students in the class selected for 

5.12 

1.42 

4.77 

1.46 

6.68** 

answers 






I liked using a SRS for tests 

5.23 

1.54 

4.83 

1.62 

29.63* 


*p<.001; **p<.005; ***p<.05. 


CONCLUSION 

The results obtained from research questions can be reduced as follows. 

1. It would seem from the comments of six instructors that incorporating student response system into their 
university level classrooms was both beneficial and enjoyable. Learning clicker technology and deciding on 
ways to incorporate technology has challenged the instructors to think about their lectures as a whole with 
respect to pacing, student interaction, and classroom engagement. 

Instructor’s perceived outcomes of the use of student response systems include increased student participation, 
increased student attendance, improved instructor-student interaction, active and collaborative learning activities, 
and an enriching educational environment. 

Subtle benefits to clickers emerged and included a decrease in the amount of time spent grading and writing 
down answers, greater understanding behind pointing out wrong answers, an increase in student generated verbal 
questions, and a new thoughtfulness behind preparing class sessions. 
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2. While instructors generally answered by saying that they did not consider the cognitive level of the their 
students when designing clicker questions, the actual slides representing clicker questions and the interview and 
observation results represent a mix of cognitive learning domains according to Bloom’s taxonomy. Many 
questions posed through the use of clickers are simply knowledge and application level questions. However, 
higher-level, synthesis and evaluation questions were represented. 

It is also obvious that a higher-level of thinking can occur as an instructor fosters discussion with the use of 
clicker questions. A question is posed, students are required to make a decision, they have to choose an answer, 
and the instructor then encourages the students to talk among themselves, generating peer discussion or peer 
learning. The student now either has support for the answer s/he chose or must defend the answer to a peer. 
Further discussion as a large group might also be generated. As the instructor guides the students through the 
process of describing why they chose the response they chose and why that response is right, wrong, or 
indifferent, this higher-level thinking can emerge. In addition, students become more likely to ask questions 
when they are able to see that they are not alone in their thinking and when they feel the instructor cares about 
their opinions and learning. Instructors can clear up misconceptions and generate class discussions about course 
content immediately on seeing student responses. 

3. While there were slightly mixed thoughts on student experiences with clickers, most of the twelve students 
interviewed in this study held a favorable regard for the use of clickers in the university classroom. Students 
noted the use of clickers associated with class credit points increased their likelihood of attending class and noted 
the improved learning that occurs on attending class. 

Students felt that being able to make peer comparisons and being able to validate their understanding of the 
course content were also benefits of the use of clickers. They enjoyed the peer discussions that instructors 
facilitated with regard to the use of clickers. Some students felt frustration over the competence level of 
instructors with clicker technology while other students were forgiving about the learning curve associated with 
the technology. 

The most beneficial activities cited by the students in this study related to clicker use were content 
comprehensions review, generation of classroom discussion, application questioning, and attendance checks. 
Students did desire that instructors use clickers daily and in various ways mainly to justify their purchase of 
clicker. The students also requested this increase in use because they felt the use of clickers did supported or 
improved their classroom learning. 

Most students in this sample were not able to make the connection between the use of clickers and the 
enhancement of higher-level thinking, but they did desire more information from their instructors as to why 
clickers were used in their classrooms. Several students were able to formulate a connection between the use of 
clickers and the improvement of higher level cognitive thinking. 

4. Student involvement, assessment, and perceived learning were examined with the help of the SRS attitude 
survey. A simple comparison indicated that male students were significantly more positive than female students 
in all three categories. Male students were more motivated and engaged when using the SRS, participated more 
in the SRS-based classrooms, liked using the SRS to test their knowledge, especially in summative evaluation, 
thought the SRS generated more class discussion, felt the SRS helped improve their learning, and overall, 
thought the SRS-based classes were better. Female students on the other hand, would seem more stressed when 
using the SRS. 

The benefits and drawbacks of the SRS were clearly reported in the literature. The research supports those 
outcomes obtained from the literature. However, the author could not find any statistical (qualitative and 
quantitative) analysis applied in any researches. It was believed that this research will elucidate the details about 
SRS and encourage other researchers to investigate various parameters affecting the success of the learning 
system. 
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